AIbase
Home
AI Tools
AI Models
MCP
AI NEWS
EN
Model Selection
Tags
Efficient tokenization

# Efficient tokenization

Swallow MS 7b Instruct V0.1
Apache-2.0
Japanese-enhanced large language model continuously pre-trained based on Mistral-7B-v0.1
Large Language Model Transformers Supports Multiple Languages
S
tokyotech-llm
48
14
Vietnamese Llama2 7b 120GB
Other
A Vietnamese-optimized large language model based on Llama-2-7B, enhanced through continual pre-training on 124GB of multi-domain Vietnamese and English data for improved language understanding
Large Language Model Transformers Supports Multiple Languages
V
bkai-foundation-models
65
35
Koreanlm 3B
KoreanLM is an open-source project dedicated to developing Korean language models, aiming to address the scarcity of Korean learning resources and inefficient tokenization processing.
Large Language Model Transformers Supports Multiple Languages
K
quantumaikr
33
2
Koreanlm
KoreanLM is an open-source language model project specifically optimized for Korean, designed for Korean grammar and vocabulary characteristics, providing efficient tokenization solutions
Large Language Model Transformers Supports Multiple Languages
K
quantumaikr
59
28
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
English简体中文繁體中文にほんご
© 2025AIbase